Revisiting User Simulation in Dialogue Systems. Do we still need them? Will imitation play the role of simulation?

Recent advancements in the area of spoken language processing and the wide acceptance of portable devices, have attracted significant interest in spoken dialogue systems. These conversational systems are man-machine interfaces which use natural language (speech) as the medium of interaction. In order to conduct dialogues, computers must have the ability to decide when and what information has to be exchanged with the users. The dialogue management module is responsible to make these decisions so that the intended task (such as ticket booking or appointment scheduling) can be achieved. Effective functioning of a dialogue system depends on the quality of the strategy used for making these decisions. Thus learning a good strategy for dialogue management is a critical task. In recent years reinforcement learning-based dialogue management optimization has evolved to be the state-of-the-art. A majority of the algorithms used for this purpose needs vast amounts of training data. However, data generation in the dialogue domain is an expensive and time consuming process. In order to cope with this and also to evaluate the learnt dialogue strategies, user modelling in dialogue systems was introduced. These models simulate real users in order to generate synthetic data. Being computational models, they introduce some degree of modelling errors. In spite of this, system designers are forced to employ user models due to the data requirement of conventional reinforcement learning algorithms. As part of this manuscript, a set of sample efficient Approximate Dynamic Programming and Kalman Temporal Differences class of algorithms are adapted to dialogue optimization. Experimental results indicate that these algorithms are indeed sample efficient and te l-0 08 75 22 9, v er si on 1 21 O ct 2 01 3 can learn optimal dialogue strategies from limited amount of training data when compared to the conventional algorithms. As a consequence of this, user models are no longer required for the purpose of optimization, yet they continue to provide a fast and easy means for quantifying the quality of dialogue strategies. Since existing methods for user modelling are relatively less realistic compared to real user behaviors, the focus is shifted towards user modelling by means of inverse reinforcement learning. Using experimental results, the proposed method’s ability to learn a computational models with real user like qualities is showcased as part of this work. te l-0 08 75 22 9, v er si on 1 21 O ct 2 01 3